Sonimime: Sonification of Fine Motor Skills

This paper describes the design of SoniMime, a system for the sonification of hand motion. Among SoniMime’s applications is the use of auditory feedback to refine fine motor skills in a wide variety of tasks. Our primary sonification method involves mapping movement to timbre parameters. Specifically, we explore the application of the tristimulus timbre model for the sonification of gestural data, working toward the goals of assisting a user to learn a particular motion or gesture with minimal deviation. We also explore real-time timbre shaping and its possible use as an intuitive sonification tool related to formant synthesis and human vowel recognition. SoniMime uses two 3-D accelerometers connected to an Atmel microprocessor which outputs OSC control messages. Data filtering, parameter mapping, and sound synthesis take place in Pd running on a Linux computer.